Papers with harmfulness discrimination
How Jailbreak Defenses Work and Ensemble? A Mechanistic Investigation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Jailbreak attacks, where harmful prompts bypass generative models’ built-in safety, raise serious concerns about model vulnerability. |
| Approach: | They propose to reframe the standard generation task as a binary classification problem to assess model refusal tendencies for both harmful and benign queries. |
| Outcome: | The proposed defenses improve model safety or optimize the trade-off between safety and helpfulness. |